Relational presentation of communications and application for transaction analysis

ABSTRACT

The system provides a method of sorting and presenting messages in a way so that the relationship in message threads can be easily observed and related messages can be identified. The system provides a way to view messages and map message threads and inboxes in two and three dimensions so that the content of messages can be easily reviewed and the relationship between messages can be seen and followed. The system is not limited to email messages but can present the relationship between multiple types of communications including emails, instant messages, texts, tweets, bulletin boards, wikis, blogs, voice conversations postings on social networks and other types of communications. In addition, the system allows for the inclusion of transactional information, including financial transactions, physical movement, asset deployment, or other acts or activities that may be related to, or independent of, the communications.

This patent application is a continuation of U.S. patent applicationSer. No. 16/226,738 filed on Dec. 20, 2018, which is a continuation ofU.S. patent application Ser. No. 12/835,724 filed on Jul. 13, 2010,which also claims priority to U.S. Provisional Patent Application No.61/225,539 filed on Jul. 14, 2009, now expired, and to U.S. ProvisionalPatent Application No. 61/262,456 filed on Nov. 18, 2009, now expired,both of which are incorporated by reference herein in their entireties.

BACKGROUND OF THE SYSTEM Field of the Invention

The invention relates generally to a system of sorting and presentingcommunications and for analyzing and tracking transactions.

Background of the Invention

Email has become an indispensable communication tool. There are a numberof email applications used for sending and receiving email. Examplesinclude email clients such as Microsoft Exchange, IBM Lotus Notes,Microsoft Outlook, Eudora, Mozilla Thunderbird, and GnuMail, and browserbased email systems such as YahooMail and GoogleMail.

As email has supplemented and replaced prior communication techniques,such as posted mail, messenger delivered communication, and otherphysical based written communication techniques, its importance asevidence in transactions has grown. Particularly in litigation, due inpart to the impact of Sarbanes-Oxley and other corporate governancerequirements, the preservation and production of email is required inevery jurisdiction.

The Federal Rules of Civil Procedure have been expanded to coverelectronically stored information (ESI) including emails. This type ofinformation must be preserved and produced in a controversy as well asin the normal course of corporate governance. A difficulty in producingemail is the need to produce all the email required and only the emailthat is required of a party to produce. Failure to produce all requestedemail can result in potential loss of rights and/or penalties fromgoverning bodies such as the SEC. Producing more email than is requiredcan breach confidentiality and put a litigant or, company in theposition of revealing data that would otherwise have remainedconfidential.

In the past, the culling, sorting, inspection, and producing of all andonly the correct emails has been time consuming and expensive. Prior artemail systems are not designed with litigation production in mind.

Another disadvantage of prior art systems is the difficulty in analyzingnon-communication transactions that may be related to communicationssuch as email, instant messaging, texting, blogs, wikis, bulletinboards, twittering, phone calls, postings on social networks such asFacebook, My Space, etc. or other types of communication.

SUMMARY OF THE SYSTEM

The system provides a method of sorting and presenting communicationssuch as email messages, bulletin board messages, SMS messages, Twittermessages, social network communications and other communications in away so that the relationship in communication threads can be easilyobserved and related messages can be identified. The system provides away to view messages and map message threads in two and three dimensionsso that the content of messages can be easily reviewed and therelationship between messages and their senders/recipients can be seenand followed. The system allows a user to enter into a message thread atany point and to then produce a visualization of the related threads,messages, and attachments associated with each individual message. Aunique tool allows the navigation of message databases with ease. Thesystem is not limited to email messages but can present the relationshipbetween multiple types of communications including emails, instantmessages, texts, tweets, wikis, bulletin boards, blogs, voiceconversations such as phone calls, postings on social networks such asFacebook, My Space, etc. and other types of communications. In addition,the system allows for the inclusion of transactional information,including financial transactions, physical movement, asset deployment,or other acts or activities that may be related to, or independent of,the communications.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a two dimensional view of the presentation of an email threadin one embodiment of the system,

FIG. 1B is a color version of FIG. 1A.

FIG. 2A is a three dimensional view containing several email threads,one of which is similar to the presentation of FIG. 1.

FIG. 2B is a color version of FIG. 2A

FIG. 3 is a diagram illustrating the thread analysis of the system.

FIG. 4 is a flow diagram illustrating the operation of an embodiment ofthe system.

FIG. 5 is a flow diagram illustrating the use of the system foranalysis.

FIG. 6 is a block diagram of an embodiment of the system.

FIG. 7 is a block diagram illustrating a system for collecting data inan embodiment of the system.

FIG. 8 shows the clipboard of the system.

FIG. 9 illustrates a system, for visualization in an embodiment of thesystem.

FIG. 10 is a 3D view with a message opened and a note attachment entryfield opened.

FIG. 11 illustrates embodiments of the trident navigation tool in anembodiment of the system.

FIG. 12 illustrates the elements of email entries.

FIG. 13A is an example of medical tracking in an embodiment of thesystem.

FIG. 13B is a version of FIG. 13A.

FIG. 14 is a flow diagram illustrating operation of analysis in anembodiment of the system.

FIGS. 15A and 15B illustrate the display of analysis data in anembodiment of the system.

FIGS. 16A and 16B illustrate a second state of analysis data in anembodiment of the system.

FIG. 17 is a detailed view of the transactions layer of FIG. 16B.

FIG. 18 illustrates document tracking in an embodiment of the system.

FIG. 19 is an example computer system in one embodiment.

DETAILED DESCRIPTION OF THE SYSTEM

The present system provides a method of maintaining and displayingrelationships between communications, and of viewing thesecommunications. In one embodiment, the system is described in connectionwith email communications. However, the system has equal application toany type of communications such as instant messages, text messages,tweets, bulletin boards, blogs, wikis, voice communications, socialnetworks, financial transactions, travel and geographic location, andthe like.

Referring to the example of entails, as entails are created, sent,and/or received, whether as sender, addressee, cc, or bcc, the systemmaintains information and linkages through the various email threads.When desired, a user may display a particular email message thread in away that can graphically represent the history of the thread and therelationship of the timing and participants in the thread. The systemmay use color coding to represent various actions and relationshipsfound in the thread. In one embodiment, the system can present a threadin a three dimensional presentation to more easily allow a user to seerelationships. Although the example described below is directed toemail, the system has equal application to bulletin boards, blogs,social networks, SMS message strings, document revision strings,software code revision strings, web content analysis strings (such asWiki's), and other environments where there may be one or more threadsof related content or other interrelationship of content. The system maybe used with other forms of interrelationships. Any form ofcommunication or transaction can be represented and analyzed by thesystem. Examples of parties can be a sender, recipient, bank account andthe like. Examples of objects can be: money, e-mail, bulletin boardpostings, social network postings, transportation, and the like

FIG. 1 is a two dimensional analysis of a thread using an embodiment ofthe system. In one embodiment, all message threads begin with a rootmessage that is the original message in the thread. In the embodiment ofFIG. 1, the root email message is disposed in the lower left of the twodimensional presentation. (Note that the root email message can just aseasily be disposed in some other location, such as the upper left of thetwo dimensional representation, and corresponding changes made in theordering of the remaining messages in the thread.) The systemillustrates a column for each sender/receiver participating in thatemail thread. In the example shown, there are five columns illustrated,but the system is not limited to this number.

Column C1 is the sender of the root message. Columns C2-C5 are columnsfor each recipient, cc, bcc, or forwarded recipient of the root messageand other messages in the thread. The image shown in FIG. 1 is at aparticular moment in time. Further activity in the thread would lead toadditional messages and/or columns being displayed.

In one embodiment of the system, a root message is illustrated in aparticular color (e.g. dark red) so that it can be quickly foundvisually. In the embodiment of FIG. 1, other messages within the samethread that are sent by the sender of the root message appear in thesame particular color (e.g. also dark red). To indicate that a recipienthas received a copy of the email from the root sender, a rectangular boxcolored in a lighter shade (e.g. light red) of the particular colorchosen for the root sender is placed in the column representing eachrecipient, cc, bcc, or forwarded recipient (for example see columns C2,C3, or C4 or rectangles 107, 108, or 109). A “To” recipient (the namedparty in the “To:” box of an email) is shown in another color, a cc orbcc is shown in another color, a reply is shown in yet another color,and a forward operation in yet another color. If desired, these colorscan be customized by the user as desired or may be limited to a defaultselection.

In one embodiment, the system defines three types of participants, root,active, and passive participants. A root participant is the originatorof the root email and is shown in dark red. Any sub-roots (first emailin a sub-thread) originated by the root participant is also shown asdark red. To indicate that a recipient has received a copy of the emailfrom the root sender, a rectangular box colored in a lighter shade (e.g.light red) of the particular color chosen for the root sender is placedin the column representing each recipient, whether they are a To, cc,bcc, or forwarded recipient. An active participant is any recipient thatreplies to a message and is shown in dark blue. To indicate that arecipient has received a copy of an email sent from an activeparticipant, a rectangular box colored in a lighter shade (e.g. lightblue) of the particular color chosen for active participants is placedin the column representing each recipient, whether they are a To, cc,bcc, or forwarded recipient. A passive participant is any recipient thatdoes not respond to any message or create any message and is shown inlight orange/tan.

In the example of FIG. 1, the participants in column C2, C3, and C4 areall active participants as they have each originated at least onemessage in the thread. The user in column C5 is a passive participant inthe example shown. This embodiment of the system shows messages as rowsin the two dimensional image of FIG. 1. In other words, the root message101 is on the bottom left and the recipients of that message are shownby lighter shade red rectangles 107, 108, 109 on the same row to theright of the root message. Here the message was directly addressed (the“To:” line) to the users in columns C2-C4 and the user in column C5 wasjust a cc'd party, In this embodiment, a cc'd party has a cc label 102indicated on the message instance.

The original message resulted in a reply from the user in column C2, asshown by dark blue message 103 on the second row from the bottom. Thismessage was directed to the users in columns C1, C3, and C4, while theuser in column C5 was not cc'd.

The third row up illustrates a number of useful features of the system.The top message in 104 is a reply by the root participant to the activeparticipant in column C2's reply message 103. Looking to the right ofthe reply message (top message) in 104 we see that it was addressed tothe user in column C2 and cc'd to the user in column C3. But in additionto representing the reply message itself, the message can be expanded toshow all of the previous messages in the thread in that single location.Message 103 is shown just below the reply message (top message) in 104and the original root message 101 is shown just below that. The systemcontemplates where there are a larger number in columns and rows at anymessage location and allows the user to gather up all of the messagespreceding any message simply by clicking on the special navigationbutton available on each message. An example of the navigation tool 110is shown in full on the top right of the display but it is alsoavailable as part of each message header in the table. As can be seen inFIG. 1, the navigation tool 110 only displays the possible actions inthis embodiment, as the tool in column C1 lacks a “left” arrow buttonsince there are no messages to the left of that column. As can be seenin FIG. 1, the navigation tool 110 only displays the possible actions inthis embodiment, as the tool in column C1 lacks a “left” arrow buttonsince there are no messages to the left of that column.

The navigation button provides new navigational options for the user. Asingle click on the double up arrow on the navigation button containedin an email within the thread advances the user to the next email above(e.g. the email in the thread immediately following). A double click onthe double up arrow moves one email down from the current email (e.g.next older). Clicking on the left arrow or right arrow allows the userto slide left or right as desired. The equal sign may be single clickedto expand or compress a chain. In one embodiment the equal sign istoggled so that alternate clicks expand and contract. In otherembodiments, a single click expands and a double click contracts.

The result is that in the table of FIG. 1, messages may appear inmultiple locations depending on the actions of the navigation tool anddesire of the user. This allows the user at any location to click for apresentation of the thread of messages that led to the message underreview, even as those messages might appear elsewhere in the table.

The system provides navigation to select the level of detail shown foremail messages in the thread analysis GUI. The system's default view canbe set to any of the following three options: (i) message header only(thin rectangle bar showing just the sender and date/time); (ii) messageheader including Subject header line; (iii) message header, Subjectline, and first two lines of text of the message. For example, supposethe system's default view is set to option (i), so that each email isrepresented by its message header only. A single-click anywhere on themessage header reveals the Subject header line. A single-click anywhereof the message header or subject line reveals the first two lines of thetext body of the message. A single click anywhere on the message expandsit further to show the full text of the email body. In other words, asingle-click on any email within the thread expands that email messageto the next level of detail. A double-click anywhere on an expandedemail message contracts to the preceding level of detail.CTRL+single-click anywhere on a fully or partially expanded emailcontracts it all the way to the system's default starting level ofdetail (in the preceding example, it would contract all the way back toshowing only the message header).

Continuing with the example of FIG. 1, we see another dark red message105 immediately above message 104. This message is a second reply tomessage 103 but with a different addressee, namely the user in columnC4. The coloring and visual presentation of the system allows a view ata glance to see that there were two replies originating from a singlemessage. In addition, date indicators are shown so that the timing ofthe messages can be seen or separated by day. Often a litigationdiscovery request will be date delimited, So the system allows the userto easily determine which messages fall into the appropriate timeperiod.

FIG. 2 illustrates a three dimensional presentation of a plurality ofthreads in an embodiment of the system. The thread based on root message101 is shown in the foreground of FIG. 2 moving from front to back.Users in columns CI-CS are shown as separate slices along theparticipants axis 200. Also shown in this view are other related threads201 and 202. The user is free to click on any thread or message in thevisualization and to move between a two dimensional type view of FIG. 1or the three dimensional view of FIG. 2.

In one embodiment of the system, the user can select one or more threadsor messages˜both contiguous or non-contiguous, and perform a number oftasks on those selected messages, such as saving to a file, printing,sharing messages to other analysts or other desired activity.

FIG. 2 includes a horizontal slider 207 on the X axis that rotates the3D cube about a vertical axis and a vertical slider 208 that rotates thecube about a horizontal axis. The origin (i.e. the starting message of athread) may be in the top right corner (or any other corner or in factin any selected position).

FIG. 2 includes the trident tool 203. A search box 204 allows the userto search for words or phrases in the data being analyzed. Theembodiment of FIG. 2 also

includes a region 205 with some filtering tools such as advanced search.root participants, date, and keywords that allow filtering andcustomization of the view presented to the user. The clipboard 206 isshown as a tabbed space in FIG. 2.

The system can provide multiple views of the 3D cube and other actionsin addition to rotation can be performed on the cube. The layers may beselectively removed and/or reordered. The system can “drill down” byremoving the top layer and moving further into the screen. The time,thread, and/or participant axes can be scrolled as desired. There mayalso be an indication of color or icon that indicates that moreinformation is available but not presented on the screen.

FIG. 3 illustrates a method of tracking messages and relationships in anembodiment of the system. The root message 101 is defined as message 1-1(representing participant 1 on the x-axis (participants) and message 1on the y-axis (relative time)). A two dimensional grid is populatedbased on the number of participants and can grow as the participants andmessages increase.

Each message is tagged with information representing the actions takenwith that message. For example, message 101 is shown with tags 2-1, 3-1,4-1; and 5-1 cc indicating that the message was directly addressed tothe users in columns C2-C4 and cc'd to the user in column C5. The replymessage 103 of FIG. 1 is tagged as 2-2 in FIG. 3. Each reply messagealso tags the messages that led to that message so that they can beexpanded on demand. Message 103 is tagged as 2-2 because it is sent fromparticipant 2 and is the second message in the chain. Messages 104 andIOS show what happens when a user sends two different replies to asingle message.

FIG. 4 is a flow diagram illustrating the operation of an embodiment ofthe system. At step 401 the user initiates the analysis system. Atdecision block 402, the system determines if it is a new analysis. Ifso, the system proceeds to step 403. If not, meaning that it is apre-existing analysis, the system proceeds to step 406.

If it is a new analysis at decision block 402, the system proceeds tostep 403 where the user is presented with a parameter input screen tohelp the user define and

refine a search or analysis. The user can search for existing and savedreports or may search for specific email threads.

At step 404 the user is presented with search results and chooses one ofthe threads for analysis. At step 405 the user can determine the scope,parameters, tags, and/or notes to be included in a report.

After step 405, or pre-existing analysis was chosen at decision block402, the system proceeds to step 406. At step 406, the user is presentedwith the main interface mode of the thread analysis system. The user canselect threads from options provided in an interface. At step 407 theuser can elect to rotate the display along the X, Y and/or Z axis andcan select certain emails to copy to a clipboard for later retrieval.This is helpful for responding to discovery requests as the user can seea graphic representation of a thread analysis and simply selectresponsive emails within that view, without the need to switch toanother mode for collection.

At step 408, the user can zoom into a view of any part of the cube formore detailed analysis. Even in the zoomed mode, all tools andinterfaces are still available to the user. At step 409, the user ispresented with a two dimensional view, as in FIG. 1. Within this view,the user can view individual emails at step 410 or attach notes to eachthread and/or email at step 411. At step 412, the user can select a newthread and/or view by using an options interface. At step 413, the usercan generate a report of all the emails or from previously tagged orclipboarded emails.

Information and Transactional Analysis

One feature of the system is the ability to maintain and display therelationships between communications activity and secondary informationand/or other transactions (i.e. financial transactions, stock trades,travel, and the like) so that complex behavior patterns, includingillicit or illegal activities, or any activities of interest, can bediscovered and tracked.

There are many circumstances where it is important to know certaininformation such as:

1 Understand which people knew about issues when the issues occurred.

2. Understand when those people learned what they knew

3, Understand how people disseminate the information they obtained.(Such as confidential information, privileged information or informationthat may have been “shaped” for specific purposes by the user.)

4. Understand what people did with the information they had (such asillicit maneuvers, financial gain, and/or criminal acts).

FIG. 5 is a flow diagram illustrating the steps that an analyst wouldtake when tracking this information. At step 501 directives and taskingis defined. This could involve indentifying individuals or entities anddetermining if they are participating in certain transactions. Forexample, the investigation could see if individual A is tradingpharmaceutical stock. At step 502 background research is performed toestablish possible persons and entities to be analyzed and examined aswell as understand additional context. At step 503 the source of usefuldata is identified, such as email accounts, telephone accounts, instantmessage accounts, twitter accounts, bank accounts, stock accounts,social networks, etc. At step 504 the analysis tools of the system areutilized. This means that the data from the identified sources isprovided to the system for display. The display can be sorted, filtered,rotated, and manipulated to reveal patterns and connections that mightotherwise be hidden. At step 505 one or more reports can be generatedthat represent threads of communications and transactions that reveal auseful pattern. At step 506 the data, reports, and analysis can beshared with others.

The system allows patterns of communication to be revealed andassociated with other behavior. There may be chains of behavior linkedto email, text, instant messages and/or phone messages. The presence ofa communication and a subsequent activity allows an inference that anactor in a transaction had knowledge or awareness of information priorto acting on the knowledge.

The system not only allows presentation of the communications andtransactions for analysis, but allows these to be shared with otheranalysts. The system in one embodiment uses preparation, messageprocessing, and visualization to accomplish the analysis.

The Preparation phase establishes “connectors” to the system. The systemautomatically connects to communication services such as MicrosoftExchange, voice message servers, and other communication systems. Theseconnections can be established by direct loading onto an enterpriseserver or via a web-based system. Once connected, the system extractsand transforms data into a form suitable for use as well as loading thesource data into a database for recall as needed.

The Message Processing phase performs semantic and metadata processing,as well as thread reconstruction, which is described in more detailbelow. The system will also perform de-duplication, identify and accountfor common users, and handle multiple email, IM, social network, andother accounts.

This visualization phase includes the visualization tools where the 2Dand 3D representation of the data so that patterns can be divined. Thisphase also includes the filtering and sorting tools so that analysis canbe enhanced. This information can be shared and reports can be authored.

When displaying the data and showing the time-based relationship betweenevents, the system can, in one embodiment, normalize the temporaldimension. In this manner, the sequential relationships are displayedmore clearly than if the temporal dimension was not normalized. Whenthere is a large time gap between events, a display withoutnormalization could disseminate out more than would be comfortablyvisible on a typical display, masking or obscuring some of therelationships that need to be seen. There are situations when the actualtemporal scale is desired, to show the real time lapse between cause andeffect, but the system allows the user to choose between actual andnormalized temporal displays.

The 3D display of the data shows multidimensional space and allows theuser to evaluate visually, using position and color, the richness of thecommunication. Multiple threads and multiple participants can be viewedin context with an infinite number of visual planes for analysis. Theuser of the system can determine which threads, which users, and canidentify which planes (i.e. communication, social network, transaction)are to be included in the display. The 3D cube presentation can bezoomed, rotated, and explored in a variety of custom presentations. Thezoom feature can be implemented via interface menu commands, or even thescroll wheel of a computer mouse (i.e. rolling forward to zoom in androlling back to zoom out).

The analyst tools in working with the system include the ability to tagand apply notes to any item or plane in the display. The user can copycollections of communications, threads, and transactions to a clipboardfor later retrieval. The displays, annotations. clipboards and otherdata can be easily shared with other analysts, exported as reports toother programs such as spreadsheets or Microsoft Word documents, and canbe statistically summarized. For example. words or sentences that repeatthroughout selected threads can be identified and counted to showpatterns, or communication patterns statistics can be analysed.

Clipboard

The clipboard 206 of FIG. 2 is shown in more detail in FIG. 8. Whenexpanded, the clipboard 800 shows graphical representations of datacopied to the clipboard. In the example of FIG. 8, there are threeelements 801-803 stored in the clipboard. Each element represents a rootemail of an email thread and may represent the entire string ofcommunications and transactions associated with that root thread. Anymessage appearing in the 2D or 3D representation may be dragged to theclipboard. The clipboard is capable of holding a root email, a responseemail, a thread, a note, and any other object in the system, includingtransactions.

FIG. 10 illustrates the 3D cube with a message 1001 selected and openedto the user (via CLICK se Trident button equal sign). (It should benoted that although the description refers to “clicks” to activatenavigation events, the system is not limited to clicks but may beenabled by any type of action or event without departing Brom the scopeand spirit of the system). The message window may be closed usingCLICK+Trident button equal sign or by clicking on the red circle with an“x” inside in the upper right corner of the message window. The text ofthe opened message can be navigated vertically using the scroll baralong the right side. The user has enabled a note annotation entry field1002 so that the user can attach comments about the email for later useand analysis. The text of the note can be vertically navigated using thescroll bar along the right side of the note.

When copied/pasted into a report document, or exported, the depictioncan remain “live” via a web-based linkage back to the source system. Inother words, for certain report formats, the reader of the report canclick into the exported image area and begin to interact with thediagram as needed to rotate or otherwise view the data.

Iconography

FIG. 12 illustrates the elements of email entries in an embodiment ofthe system. Message 1201 is an example of a sender message. The message1201 includes the name of the sender 1202, the date/time 1203, and anindicator 1204 when there is an attachment (this indication alsoincludes a numeral representing the number of attachments). The tridentnavigation tool 1205 is present on each message, along with the firsttwo lines 1206 of the message and the root subject 1207. Message 1208 isan active participant's response added to an email chain. Message 1210is a “cc'd” received root email which in this example includes anindicator 1209 that it is a “cc” copy versus the addressee. Message 1211is a cc recipient of the active participant's email response. Thesemessages are color coded as noted above with respect to FIG. 1.

Description of Trident Expand/Collapse Functionality

The “Expanded email stack” means an email message cell that has beenexpanded to show all the emails up to that point in the thread; if theuser has highlighted an email message within the thread, then SHFT+clickon Trident button DOWN arrowheads will expand the “stack” of emails thatrepresent the thread up until that point. The trident then would takethe form of the left trident of FIG. 11

SHFT+click on Trident button's DOWN arrowheads-->expand the stack toshow all the emails within the chain up until that point in the thread;the emails are shown in truncated form (sender, date and Subject1-leader only)

SHFT+Trident button's DOWN arrowheads again-->goes to the next level ofdetail for each message (sender, date, and Subject Header, add first 2lines of text of the body of each email message within the stack)

SHFT+Trident button's DOWN arrowheads one more time goes to the nextlevel of detail for each message (same as above, but add the full textbody of each message within the stack)

SHFT+Trident button UP arrowheads: contracts the stack of email messagesin the reverse order to what SHFT+double down arrowheads does.

(a possible future modification: SHFT+Trident equal sign fully collapsesor fully expands the stack immediately (don't have to expand/collapsethe email messages in steps).

In one embodiment, the trident tool can be adjusted to skip a certainnumber of threads or messages (e.g. 10 at a time) to speed up navigationthrough the data. This would be achieved using the CRIL+Trident UP orDOWN arrowheads and the trident would appear as on the right in FIG. 11.

Looking at FIG. 1 a user can click on the right arrow on a Tridentbutton in email message 103 or 107 (within column C2), The correspondingaction would be that the entire column to the right (C3) would slide“under” C2 and be hidden. To hide the next column (C4), you click on theright arrow again. CTRL+right arrow would expand all the hidden columns.

The left arrow has similar function (but for columns to the left of thecolumn you are currently interacting with).

Thread Management

After a data set has been provided to the system, such as by importingdata from an email exchange, bulletin board database or bulletin boardcollection system, social network database, instant messaging database,etc, the system does the following to sort, identify, and createrelationships.

De-Duplication:

Go through each message in the data set and examine the Message-Id:header. Import each message with a unique Message-Id: into the database.(Although the system is described in an example using emails, the systemhas equal application to other communication types such as thosementioned herein).

Initial Threading:

For each message in the database, create a thread with the thread namethe same as the subject of the message.

For each message in the database, examine its Message-ID: header andscan all other messages in the database for References: or In-Reply-To:headers. If a reference is found to the message-id and the messages arein different threads, join the threads by setting the thread id equalfor all messages referencing the message-ID. More than one reference maybe found to the message-id so more than one thread may need to bejoined.

Threads with a Missing Link (Split Thread):

A split thread is when two messages in the same thread point to a thirdmessage that is missing from the database. It may be missing from thedatabase because it was intentionally deleted or it may have been storedon a server that is not included in the dataset (messages on a partneror competitor's servers). This often occurs if a message is sent to athird party who discusses it internally and then replies.

For each message in the database, examine its References: andIn-Reply-To: headers. For each of the values in these headers that donot have a corresponding Message-Id: in the database, search thedatabase again for different messages that have the same References: or.In-Reply-To: id, Join threads in which the two messages refer to thesame message but are in separate threads.

After joining split threads it may be necessary to repeat step 2 of theinitial threading process to join threads that could not be combined dueto the missing message link,

Inferred Thread Relationships:

Some threads may still be broken after processing missing links.Remaining ways to combine threads are:

Message content analysis

Message subject collapsing

Threads with the same subject and participants:

In one embodiment, the system makes an assumption that a participant whosends two messages with the same subject within 24 hours isparticipating in the same thread. This may occur when someone makes anannouncement to a large group, receives questions, and then issues afollow-up to the message to address those questions.

For the purposes of this analysis, subjects are considered identicalafter any “Fwd:” or “Re:” prefixes are removed.

Participation requirements should be tuned so that the most number ofcorrect identifications occur with the least number of falseidentifications. Variables could include:

Extending time criteria to messages within one week

Match messages with the same subject where participant matching can besender or receiver

Threads with the Same Content:

Message textual analysis is complex. It compares all of the text in amessage with ail of the text in another message. A similarity measure iscalculated, such as number of words (phrases, or sentences) matcheddivided by total number of words in the message. If the similaritymeasure exceeds a threshold, the messages are considered to be in thesame thread.

This method is interesting in that it may identify connections betweenvery unrelated threads and help understand how information is beingdisseminated. For instance, if an executive sends a message to managersthroughout the organization describing “the upcoming project Barnstorm”these messages could be linked to threads from employees who should nothave received that information.

At the same time, this method suffers from its inability to joinmessages that do not include the same language. This may happen if amanager sends a request to an employee asking for a timecard and theemployee creates a new message with only an Excel document in it anddoes not include a copy of the manager's request.

System Block Diagram

FIG. 6 is one embodiment of a block diagram of an embodiment of thesystem. The system obtains data from sources such as file servers 601,email servers 602, and web servers 603. These sources provide datasources 604 such as instant messages, email, common documents, versioncontrol repositories, internal wikis, social networks, etc. A processingengine 605 includes a metadata filter 606, semantic processor 607, andtext conversion engine 606 to extract information and relationships fromthe source data and provides it to relational database 609. This data isprovided to visualization and discovery application 610 that controlsthe display and representation of the information. This providesinformation to the case preparation application tool 611 for reportgeneration, sharing, etc.

FIG. 7 is a block diagram of an alternate embodiment of the system. Thesystem operates in a plurality of stages. Stage. 701 is thecommunications stage. Stage 702 is a metadata processing stage and isfollowed by a semantic processing stage 703. The system includes arelationship building stage 704 and indexing stage 705.

The communication stage 701 is where raw data is collected forimportation to the rest of the system. The communication stage may takedata from a plurality of sources, such as email 706 through emailimporter 707, BBS information 708 via bulletin board importer 709,social media data 710 via social media importer 711, and message data712 through instant messenger importer 713.

Email data 706 may be from any of one or more email system and may be informats such as .pst files, Gmail files, .nsf (Lotus Notes) files, andthe like. The email is imported with any attachments that may beassociated with the emails. Bulletin board data 708 may be messages orpostings from web based bulletin boards and may come from linear orthreaded (and sub-threaded) BBS sources (e.g. usenet) using bulletinboard engines such as, for example, vBulletin. Bulletin board messagesor postings are imported with any attachments or imbedded media such asvideo clips, photos, links, audio clips.

Social media data 710 may include Twitter postings, Facebook wallpostings or any other message based communications on social media, suchas comments to videos or blogs and associated comments. Social Mediamessages or postings are imported with any attachments or imbedded mediasuch as video clips, photos, audio clips, links, etc.

Messenger data 712 includes AIM, SMS, Jabber, MSN, Blackberry Messenger,or any other instant message systems. Messenger messages are importedwith any attachments or imbedded media such as video clips, photos”audio clips, links, etc.

Communications from each of these sources may be processed as a batch,or in a continuous manner as data arrives from each source, such as livemonitoring of communications. Communications may also arrive multipletimes or out-of-order, as they are placed in order and de-duplicated bythe indexing engine 719.

The semantic processing stage 703 comprises an entity extractor 715 anda similarity engine 716. The entity extractor 715 determines locations,names, places and other noun-phrase concepts which may be optionallyguided by a user defined wordlist, related to each message orcommunication. The similarity engine 716 uses the concepts identified bythe entity extractor to identify related documents. This processconducts a search for other documents that have locations. names,places, or other noun-phrase concepts similar to the document beingingested. If the number of matching concepts exceeds a threshold thedocuments will be identified as similar, for use in later processingsteps.

The relationship builder stage 704 includes a profile builder 717 and aconversation builder 718. The profile builder 717 identifiesparticipants and any aliases, addresses, names, related accounts or usernames, and other identifying information to identify a participant aswell as possible regardless of the account that is being used. Theconversation builder 718 analyzes documents to determine if they arepart of a new or existing communications thread. The conversationbuilder uses the output of the similarity engine 716 and the techniquesdescribed herein. This process incorporates a feedback loop so that eachnew document added to a thread helps to identify additional candidatedocuments. The conversation builder 718 is helpful when responding todiscovery requests by allowing appropriate responses to be identifiedeven if the communications do not share a common subject line, thread,or topic.

The indexing stage 705 includes an indexing engine 719 to eliminateduplicates and to annotate and index each message. The indexed andprocessed communications and messages are stored in database 720 forlater retrieval and display.

Visualization

FIG. 9 is a block diagram illustrating an embodiment for generatingdisplay visualizations of data using the system. In this embodiment ofthe system there are a number of stage in the visualization process,including Query 901, Refine 902, Assemble 903, Navigate 904, andVisualize 905.

The Query stage 901 comprises a search engine 906 to allow specific datato be pulled out of the database 720. The search defines certainparameters such as date range, participant, keyword, and other facets ofthe search. These other facets can include bank accounts, locations,travel, transfers, which message sources to use, and other filteringoptions. The analyst will construct a search query using these limitersand advance to the refine stage 902. The search may be performed usingany search capabilities offered by the search engine, such as thoseindicated above or semantic search, proximity search, lemmatization,Boolean operators, etc.

The Refine stage 902 comprises a Refine Limit Pivot capability 907. ThePivot capability 907 implements a pivot table which allows the analystto select a new search based on additional concepts found in the currentsearch. The Refine capability allows the analyst to assign each result aquality: find more like this result and show fewer like this result. TheLimit capability allows the analyst to add additional constraints to thesearch to narrow the result set. The Refine Limit Pivot capabilityincorporates a feedback loop the search can be looped back multipletimes as desired to refine the search and increase the confidence levelof the analyst.

The Assemble stage 903 brings the data together in two ways so that theuser is presented with all options upon actual display. The system usesa grid assembler 908 to generate a 2D view of the data (as in FIG. 1)and the universe assembler 909 assembles a 3D view such as in FIG. 2.The assembly typically uses a message at a starting origin point andbuilds the display from that point, determining position and color foreach message that will be part of the visualization. In the gridassembler and universe assembler, a number of configurations for eachaxis are possible.

For the participant axis, in one embodiment, participants are orderedbased on number of messages sent. In another embodiment, participantsare ordered based on their role in the organization (executive, manager,employee) or their business unit (engineering, quality control,manufacturing). Additional embodiments allow the user to specify thetechnique used to order participants.

For the message axis, in one embodiment, messages are ordered based ontime sent in a manner where the time between messages is preserved. Inanother embodiment, messages are ordered based on time sent where thetime between messages is compressed. In another embodiment, messages areordered based on a time scale that is non-aligned and resets back to 0for each thread.

The Navigation stage 904 links the messages in the display usingnavigation engine 910 and cache 911 so that the trident navigation toolcan be used to move from message to message as described herein. TheVisualization stage 905 presents any of the possible views, for example,the 2D view 912 or the 3D view 913.

Medical Tracking

The system can be used in one embodiment for medical tracking as shownin FIG. 13. In one embodiment of the medical tracking system, eachmessage includes at least certain information, including patient ID,participant or entity, the layer (e.g. communications, reports,prescriptions, etc.), message type (e.g. medication prescription), andthe date and time. The system uses color coding or some other visualindicator to represent different types of messages in the medicaltracking embodiment.

For example, the system may use a dark color to represent an originatingrequest, and a lighter color to represent a response to that request. Inaddition, a plurality of types of communications are defined byparticular colors. This makes it possible to see at a glance what typeof communication is represented by a message in a two or threedimensional representation of data. In the example of medical tracking,red and light red represent a GP's initiating communications andresponses. Dark blue and light blue represent a specialists initiatingcommunications and responses. Dark green and light green representmeds/prescription initiators and responses. Dark orange and light orangerepresent test/diagnostic requests and responses, and dark purple andlight purple represent procedures/operations requests and responses.

As can be seen in FIG. 13, it is possible to overlay different types ofdata planes that each represent a different activity in the a patient'streatment. The messages can be time normalized so that the flow ofactivity can be seen very easily. Selecting or clicking on anyindividual message will bring up all documents associated with thatcommunication or procedure.

Financial Tracking/Intelligence

The system enables analysts to correlate communications and activitiesfrom a plurality of participants to reveal patterns and causation ofevents. In some cases these events could be crimes that have beencommitted or are in the planning stage. By presenting and correlatingdata from multiple sources, an analyst can track activities,communications and transactions of multiple participants.

In one embodiment, the system tracks a number of types of activity andcan present each type in its own visual layer. In one embodiment, theremay be a communications layer, a transactions layer, and a travel layer.This allows an analyst to see the relationship between communications,movement of funds, and location of participants.

The system can be described in one embodiment in the flow diagram ofFIG. 14. At step 1401 the analyst determines the participants of theanalysis. At step 1402 the analyst identifies the communicationsaccounts of each participant. This may include email, SMS messaging,social media, twitter, bulletin board postings, etc. At step 1403 thesystem collects the communications from these sources and extractsmetadata and other information using, for example, the system of FIG. 6.

At step 1404 the system builds a presentation layer for thecommunications of the participants. At step 1405 the analyst identifiesfinancial accounts associated with the participants. At step 1406 thesystem retrieves the account information associated with theparticipants and at step 1407 builds a presentation layer for thefinancial transactions. These transactions locations and bank accountsare based on other conversations these participants are having in otherthreads (with their financial backers, with the leader of a terroristcell, with other parties, and so forth). The system software assemblesthis information to get a confidence interval, and correlates locationand funds to the discussions. The system can import bank statements andlocation details from external databases/sources to improve thatconfidence.

At step 1408 the system collects travel and geographic information foreach participant (along with time/date information) and at step 1409 thesystem builds a presentation layer for the travel/geographicinformation. At step 1410 the system correlates and normalizes thelayers so that they can be viewed in relationship to the overallactivity of the participants. The timeline of the layers can be relativeor normalized so that cause and effect can be shown more easily.

The system identifies changes in a participant's location or bankaccount balance, and indicate those changes in a certain color. Thesystem identifies specific locations or amounts (i.e. Yemen or amountsgreater than $2 million) and indicate those in a certain color. Thesystem allows the intelligence analyst to program in suspicious activityflags using Boolean logic and functions, like the expression evaluatorin Excel. Alerts can notify the analyst whenever there's a correlationbetween databases, and indicate how closely things correlate.

The following is an example to show some capabilities of the system.

Abe is the ringleader and lives in the US.

Bonnie is the money launderer and participates in the activity.

Clyde is the financing source and lives in Japan.

Darryl is the point man on the ground.

Ed leads the terror cell and is being paid to set up and detonateexplosives.

FIGS. 15A and 15B illustrate a display that shows the state of thecommunications 1501, transactions 1502 and geographic locations 1503 ofthe participants.

In the communication layer, each of the participants has their owncolumn representing their communications. In this case, thecommunications have been filtered to show only those communicationsrelated to this particular plot. As noted previously, the messages maybe color coded to show the nature of the message and the relationshipbetween originator, recipient, responder, copied-to, etc. The financialtransactions 1502 shows two accounts of interest. In this case, there isnot direct correlation between the accounts. In other words,transactions are not made directly between the accounts, but may gothrough a plurality of middlemen. The system allows to show the causeand effect of withdrawals and deposits on the two accounts without theneed to track the transaction through all the third parties.

The geographic indicator 1503 shows the location of a participant at thetime of any particular message. By highlighting a message, the locationof each participant (if known) at the time of that message is displayedin the geographic region. This display is dynamically changed asdifferent messages are selected. The system also has the ability to“play” a scenario showing each message as it is created in time andshowing the changes to the location map 1503 and financial information1502 as the playback proceeds.

FIGS. 16A and 16B illustrate the final state of the communications 1601.transactions 1602, and geographic locations 1603 of the participants inthe plot.

communications layer 1601 is populated with a number of messages,responses, and copies related to the participants. In addition, thetransactions layer now shows three accounts of interest and differentcolors to represent the transactions. Finally, the location map showsthe final locations of the participants.

FIG. 17 is a detailed view of the financial transactions layer 1602. Inthe example shown, account B1 is a Turkish bank, account B2 is a Swissbank, and account B3 is a Mid-East bank. The system shows thetransactions overlaying the communications layer so the relationshipbetween the transactions and the communications can be seen more easily.The system also uses color or some other visual indicator to representthe state of the accounts. In this example, medium green represents anaccount, dark green represents the state when funds transfer in andlight blue represents the state when funds transfer out.

Again referring to FIG. 17, account B1 begins with a zero balance whilethe account B2 has a balance of $100 million. In response to andcorrelated with a communication from one or more of the participants, a$10 million withdrawal is made from account B2. This is represented bythe cut-out circle on the lower left of account B2 and a visible minussign. This is an additional visual indicator representing withdrawal inaddition to the change in color of account B2 after the transaction.

Account B1 then shows a new color (e.g. dark green) and a “plus” signindicating a deposit, also of $10 million. Meanwhile, the status ofaccount B3 has not changed at this time and remains, for example, mediumgreen. Subsequently, there is a withdrawal of $1 0 million for account B1 (shown by the circular cut-out and minus sign) and a correspondingdeposit of $10 million in account B3, shown by color change (e.g. darkgreen) and the plus sign circle.

If there are withdrawals from an account that has already had a priorwithdrawal, in one embodiment the second withdrawal is shown by a secondcircular cutout or indent with a minus sign. In another embodiment, thesize of the circular cut-out is enlarged to indicate multiplewithdrawals. In yet another embodiment, the system reverts the accountto the neutral color (e.g. medium green) and then changes it to lightgreen and

shows the second withdrawal in the same manner as above, with a singlecut-out or indicator.

All of the features described previously apply to this application ofthe system as well. For example, the user can mouse over or right-clickon the display to have messages appear in their full form, or to add orchange the presence or status of participants.

Document Tracking

The system also capable of providing a display for document tracking.FIG. 18 illustrates an embodiment of document tracking using the system.Three documents are represented by message icons 1801, 1802, and 1803.The visualization layer uses color coding for different versions andrevisions of documents. The user can see who received the document, whorevised it and when, latest and earlier versions, etc.

Embodiment of Computer Execution Environment (Hardware)

An embodiment of the system can be implemented as computer software inthe form of computer readable program code executed in a general purposecomputing environment such as environment 1900 illustrated in FIG. 19,or in the form of bytecode class files executable within a within aJava™ or other virtual machine run time environment running in such anenvironment, or in the form of bytecodes running in such an environment,or in the form of bytecodes running on a processor (or devices enabledto process bytecodes) existing in a distributed environment (e.g., oneor more processors on a network). A keyboard 1910 and mouse 1911 arecoupled to a system bus 1918. The keyboard and mouse are for introducinguser input to the computer system and communicating that user input tocentral processing unit (CPU 1913. Other suitable input devices may beused in addition to, or in place of, the mouse 1911 and keyboard 1910,such as a tablet PC, iPad, touchscreen device, smartphone/PDA, etc., I/O(input/output) unit 1919 coupled to bi-directional system bus 1918represents such I/O elements as a printer, A/V (audio/video) I/O, etc.

Computer 1901 may include a communication interface 1920 coupled to bus1918. Communication interface 1920 provides a two-way data communicationcoupling via a network link 1921 to a local network 1922. For example,if communication interface 1920 is an integrated services digitalnetwork (ISDN) card or a modem, communication interface 1920 provides adata communication connection to the corresponding type of telephoneline. Which comprises part of network link 1921. If communicationinterface 1920 is a local area network (LAN) card, communicationinterface 1920 provides a data communication connection via network link1921 to a compatible LAN. Wireless links are also possible. In any suchimplementation, communication interface 1920 sends and receiveselectrical, electromagnetic or optical signals which carry digital datastreams representing various types of information.

Network link 1921 typically provides data communication through one ormore networks to other data devices. For example, network link 1921 mayprovide a connection through local network 1922 to local server computer1923 or to data equipment operated by ISP 1924. ISP 1924 in turnprovides data communication services through the world wide packet datacommunication network now commonly referred to as the “Internet” 1925.Local network 1922 and Internet 1925 both use electrical,electromagnetic or optical signals which carry digital data streams. Thesignals through the various networks and the signals on network link1921 and through communication interface 1920, which carry the digitaldata to and from computer 1901, are exemplary forms of carrier wavestransporting the information.

Processor 1913 may reside wholly on client computer 1901 or wholly onserver 1926 or processor 1913 may have its computational powerdistributed between computer 1901 and server 1926. Server 1926symbolically is represented in FIG. 19 as one unit, but server 1926 canalso be distributed between multiple “tiers”. In one embodiment, server1926 comprises a middle and back tier where application logic executesin the middle tier and persistent data is obtained in the back tier. Inthe case where processor 1913 resides wholly on server 1926, the resultsof the computations performed by processor 1913 are transmitted tocomputer 1901 via Internet 1925, Internet Service Provider (ISP) 1924,local network 1922 and communication interface 1920. In this way,computer 1901 is able to display the results of the computation to auser in the form of output.

Computer 1901 includes a video memory 1914, main memory 1915 and massstorage 1912, all coupled to bi-directional system bus 1918 along withkeyboard 1910, mouse 1911 and processor 1913.

As with processor 1913, in various computing environments, main memory1915 and mass storage 1912, can reside wholly on server 1926 or computer1901, or they may be distributed between the two. Examples of systemswhere processor 1913, main memory 1915, and mass storage 1912 aredistributed between computer 1901 and server 1926 include thin-clientcomputing architectures and other personal digital assistants, Internetready cellular phones and other Internet computing devices, and inplatform independent computing environments,

The mass storage 1912 may include both fixed and removable media, suchas magnetic, optical or magnetic optical storage systems or any otheravailable mass storage technology. The mass storage may be implementedas a RAID array or any other suitable storage means. Bus 1918 maycontain, for example, thirty-two address lines for addressing videomemory 1914 or main memory 1915. The system bus 1918 also includes, forexample, a 32-bit data bus for transferring data between and among thecomponents, such as processor 1913, main memory 1915, video memory 1914and mass storage 1912. Alternatively, multiplex data/address lines maybe used instead of separate data and address lines.

In one embodiment of the invention, the processor 1913 is amicroprocessor such as manufactured by Intel, AMD, Sun, etc. However,any other suitable microprocessor or microcomputer may be utilized. Mainmemory 1915 is comprised of dynamic random access memory (DRAM). Videomemory 1914 is a dual-ported video random access memory. One port of thevideo memory 1914 is coupled to video amplifier 1916. The videoamplifier 1916 is used to drive the cathode ray tube (CRT) rastermonitor 1917. Video amplifier 1916 is well known in the art and may beimplemented by any suitable apparatus. This circuitry converts pixeldata stored in video memory 1914 to a raster signal suitable for use bymonitor 1917. Monitor 1917 is a type of monitor suitable for displayinggraphic images, such as computer monitor tablet PC, iPad,smartphone/PDA, touchscreen device, etc.

Computer 1901 can send messages and receive data, including programcode, through the network(s), network link 1921, and communicationinterface 1920. In the Internet example, remote server computer 1926might transmit a requested code for an application program throughInternet 1925, ISP 1924, local network 1922 and communication interface1920. The received code maybe executed by processor 1913 as it isreceived, and/or stored in mass storage 1912, or other non-volatilestorage for later execution. In this manner, computer 1901 may obtainapplication code in the form of a carrier wave. Alternatively, remoteserver computer 1926 may execute applications using processor 1913, andutilize mass storage 1912, and/or video memory 1915. The results of theexecution at server 1926 are then transmitted through Internet 1925, ISP1924, local network 1922 and communication interface 1920. In thisexample, computer 1901 performs only input and output functions.

Application code may be embodied in any form of computer programproduct. A computer program product comprises a medium configured tostore or transport computer readable code, or in which computer readablecode may be embedded. Some examples of computer program products areCD-ROM disks, ROM cards, floppy disks, magnetic tapes, computer harddrives, servers on a network, and carrier waves.

The computer systems described above are for purposes of example only.An embodiment of the invention may be implemented in any type ofcomputer system or programming or processing environment, such as tabletPC, iPad, touchscreen device, PDA, etc.

Thus, a system for analyzing and displaying relationships betweencommunications and transactions has been described.

What is claimed is:
 1. (canceled)
 2. A method for presenting relationalinformation, the method comprising: extracting a plurality of entitiesfrom electronically stored information; identifying a plurality ofthreads from the electronically stored information; and determining aplurality of relationships among the plurality of entities according tothe plurality of threads, wherein the plurality of relationshipscomprises a plurality of relationship types.
 3. The method of claim 2,wherein each of the plurality of threads comprises one or more messages.4. The method of claim 2, wherein each of the plurality of threadscomprises multiple types of communications.
 5. The method of claim 2,wherein each of the plurality of threads comprises transactionalinformation.
 6. The method of claim 5, wherein the transactionalinformation is independent of a communication.
 7. The method of claim 5,wherein the transactional information is related to a communication. 8.The method of claim 5, wherein the method comprises tracking thetransactional information over time.
 9. The method of claim 5, whereinthe method comprises following a relationship of the plurality ofrelationships
 10. The method of claim 2, wherein the method comprisesidentifying a document related to a relationship of the plurality ofrelationships.
 11. The method of claim 2, wherein the method comprisesidentifying a message related to a relationship of the plurality ofrelationships.
 12. The method of claim 2, wherein the method comprisesdisplaying a graph of relationships of a selected relationship type. 13.The method of claim 2, wherein the method comprises following arelationship of the plurality of relationships.
 14. The method of claim2, wherein the method comprises maintaining a time history of theplurality of relationships and the plurality of entities.
 15. The methodof claim 2, wherein each of the plurality of threads comprises relatedcontent.
 16. The method of claim 2, wherein the method comprisesanalyzing a document to determine whether it is related to a thread ofthe plurality of threads.
 17. The method of claim 2, wherein the methodcomprises adding one or more new documents to a thread of the pluralityof threads.
 18. The method of claim 17, wherein the method comprisesidentifying a different document according to one or more new documents.19. The method of claim 2, wherein the extracting of the plurality ofentities is guided by a user-defined wordlist.
 20. The method of claim2, wherein the plurality of entities are determined by a noun-phraseconcept.
 21. The method of claim 2, wherein each of the plurality ofentities is one of a location, a name and a place.
 22. The method ofclaim 2, wherein the plurality of relationship types comprises acommunication relationship.
 23. The method of claim 2, wherein theplurality of relationship types comprises a financial relationship. 24.The method of claim 2, wherein the plurality of relationship typescomprises a location relationship.
 25. The method of claim 2, whereinthe electronically stored information comprises a plurality ofcommunications accounts of each entity.
 26. The method of claim 2,wherein the plurality of entities are found in metadata of theelectronically stored information.
 27. The method of claim 2, whereineach entity of the plurality of entities is represented by a graphicalattribute other than text.
 28. A system for presenting relationalinformation, the system comprising: a memory operable to storeinformation; and a processor operable to: extract a plurality ofentities from the information, identify a plurality of threads from theinformation, and determine a plurality of relationships among theplurality of entities according to the plurality of threads, wherein theplurality of relationships comprises a plurality of relationship types.29. The system of claim 28, wherein each of the plurality of threadscomprises one or more messages.
 30. The system of claim 28, wherein eachof the plurality of threads comprises multiple types of communications.31. The system of claim 28, wherein each of the plurality of threadscomprises transactional information.
 32. The system of claim 31, whereinthe transactional information is independent of a communication.
 33. Thesystem of claim 31, wherein the transactional information is related toa communication.
 34. The system of claim 31, wherein the processor isoperable to track the transactional information over time.
 35. Thesystem of claim 31, wherein the processor is operable to follow arelationship of the plurality of relationships
 36. The system of claim28, wherein the processor is operable to identify a document related toa relationship of the plurality of relationships.
 37. The system ofclaim 28, wherein the processor is operable to identify a messagerelated to a relationship of the plurality of relationships.
 38. Thesystem of claim 28, wherein the processor is operable to display a graphof relationships of a selected relationship type.
 39. The system ofclaim 28, wherein the processor is operable to follow a relationship ofthe plurality of relationships.
 40. The system of claim 28, wherein theprocessor is operable to maintain a time history of the plurality ofrelationships and the plurality of entities.
 41. The system of claim 28,wherein each of the plurality of threads comprises related content. 42.The system of claim 28, wherein the processor is operable to analyze adocument to determine whether it is related to a thread of the pluralityof threads.
 43. The system of claim 28, wherein the processor isoperable to add one or more new documents to a thread of the pluralityof threads.
 44. The system of claim 43, wherein the processor isoperable to identify a different document according to one or more newdocuments.
 45. The system of claim 28, wherein the extracting of theplurality of entities is guided by a user-defined wordlist.
 46. Thesystem of claim 28, wherein the plurality of entities are determined bya noun-phrase concept.
 47. The system of claim 28, wherein each of theplurality of entities is one of a location, a name and a place.
 48. Thesystem of claim 28, wherein the plurality of relationship typescomprises a communication relationship.
 49. The system of claim 28,wherein the plurality of relationship types comprises a financialrelationship.
 50. The system of claim 28, wherein the plurality ofrelationship types comprises a location relationship.
 51. The system ofclaim 28, wherein the electronically stored information comprises aplurality of communications accounts of each entity.
 52. The system ofclaim 28, wherein the plurality of entities are found in metadata of theelectronically stored information.
 53. The system of claim 28, whereineach entity of the plurality of entities is represented by a graphicalattribute other than text.